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Abstract 

We consider the top-quark decay t ^ fb and t Tbxi via explicit im- 
parity violating interactions in SUSY theories. We discuss the observability 
of those channels at the Fermilab Tevatron collider. The existing Tevatron 
data indicate a 95% confidence level upper bound on the coupling to be 0.94 
(0.63) for a long-lived (short-lived) Xi with ruf = 70 GeV. At Tevatron Run 
II with an integrated luminosity of 2 (10) fb~^, one can obtain a 2a constraint 
as 0.38 (0.24) for a long-lived Xi and 0.29 (0.19) for a short-lived Xi^ beyond 

the current indirect limit. 
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I. INTRODUCTION 



Physics associated with the top quark is fascinating. Being the heaviest elementary par- 
ticle observed so far with a mass ^ 175 GeV, the top quark has the greatest potential 
to probe new physics beyond the Standard Model (SM). Examples considered include the 
radiative electroweak symmetry breaking due to the large top-quark Yukawa coupling in 
supersymmetric (SUSY) grand unified theories (GUT), and the dynamical electroweak sym- 
metry breaking in the top-quark sector. Furthermore, the fact that the top quark is about 
35 times more massive than its SU(2)-doublet partner, the b quark, makes it an attractive 
candidate for studying flavor physics. 

The Fermilab Tevatron will start a new run with significant upgrade on luminosity, 
reaching approximately 2 fb~^ in a year. This would yield more than ten thousand tt 
pairs and thus provide a great opportunity to explore new physics in the top-quark sector. 
Observing the top-quark rare decays may reveal information on new physics. For instance, 
if there are charged scalars such as the charged Higgs bosons (H^), the decay t H^b 
may occur if kinematically accessible. This search has been carried out with the existing 
data from the Tevatron by the CDF and DO collaborations |ll],0. The non-observation of the 
predicted signal has established limits on the charged Higgs boson mass and its coupling to 
the top quark. In SUSY GUT models, one may be motivated to search for decays to scalar 
quarks and gauginos 

t^iiXi and bx+ (1) 

where ii is the lighter top-squark and Xi the lightest neutralino, assumed to be the lightest 
supersymmetric particle. Furthermore, if i?-parity (Rp) is broken @,|^, then Xi iiot be 
stable and even possibly decay within the detector, leading to a different signature in the 
Tevatron collider experiments 0,0] • 

Process Eq. (|1|) may be disfavored by kinematics for heavy SUSY particles. Also, generic 
i?p-breaking {^p) interactions are subject to stringent experimental constraints due to the 
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absence of proton decay and large flavor- changing neutral currents in the lepton and quark 
sectors ||^. However, the bounds on operators involving the third generation fermions are 
particularly weak, which reflects our lack of knowledge for flavor physics in the third gen- 
eration. If there exist sizeable interactions involving the third generation, then the top 
quark may directly decay to other interesting channels. In fact, decays of the top quark and 
the top squark have been studied in scenarios where Rp is spontaneously broken [§. In this 
Letter, we explore the explicit couplings 

^333 bLb*R + h^h^V^L + bpT^rl}>L 

-^LiLb*R-h^tLfL-bj^Tj.L) +h.c. (2) 

where the subscript for the coupling indicates the third generation for all particles involved, 
and r£ is the charge conjugate fleld. The above interactions lead to the top-quark decay to 

b r+x?, (3) 
&X? r+, (4) 
6r+ X?. (5) 

Motivated by SUSY GUT models, we assume that f is lighter than the top quark and it 
decays dominantly to rxi, as indicated in Eq. @. 6 is heavier and the t —>■ br^ channel 
in Eq. (|) is a three-body decay via a virtual b*. We also take m^^ + m^o > rrit so that 
the channels of Eq. ([l|) are kinematically suppressed as we only have the three-body decay 
shown in Eq. via a virtual t*. 

There exist weak indirect bounds on the coupling A333. From the Z t~^t~ partial 
width via the ^p radiative corrections P,p^, a 2cr limit IA333I < 0.58 was obtained for 
mi = 100 GeV. Considering the decay rate of -B ^ rUrX yielded a bound of similar size 
pj| . The bounds become weaker for a heavier b. To concentrate on this possibly large 
coupling, we assume that the other ^p couplings involving light fermions are much smaller. 
The collider phenomenology then depends on whether in Eqs. to would decay 
within the detector or not, as we will discuss in details. 
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t-^bf+ 

t-^bT+ 

and t — s> tiXi 




FIG. 1. Feynman diagrams for top-quark two-body (a) and three-body (b,c) decays via -R-parity 
violation operators in Eq. (|2[). 

The rest of the paper is organized as follows: In Sec. |T|, we discuss the two-body and 
three-body top decay widths due to A333 and obtain the respective branching fractions. In 
Sec. we study the new signatures coming from these decays and the sensitivity to probe 
A333 at the Tevatron collider. We make some concluding remarks in Sec. |V[ 



II. TOP DECAYS 

We evaluate the partial decay widths for processes (H), and @ depicted in Fig. |l|. 
Denoting the two- and three-body decay widths by r2 and r3, and neglecting the masses of 
b and r, we obtain for the two-body decay 

r,(,..,).^,„,(i-;|)\ (6) 

which agrees with Ref. P]; and for the three-body decay 

with Cl,r and C'^ ^ being the left- and right-handed coupling constants for the vertices bbxi 
and tixi respectively |jl2| . The phase space integrals Jj (x) are given by 



and 
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Ii{x) = 12 f dx- ^^——r\{l + x^-x)+Ax]'^A]x^'^{l,x^,x), (9) 

JO (1 — x/XiY L ^ ^ J 

where Xi = mf/mf, A = Cj^C'^fJ {C'^ + C'^) and A is the usual Kallen kinematical function. 
The factor of 12 is to normahze the integrals Ii and Ii to unity in the limit x^o and 
xi.xi ^ 1. The ratio of these two decay widths is r2/r3 ^ 487r^(C2 + C]^)'mf/m^, which 
shows that the two-body mode dominates by more than three orders of magnitude. 
The SM partial width for the top-quark decay t bW~^ is 

The ratio for the decay width with respect to that of the SM is thus 

T2/TSM ^ 2|A^33pM^/^7V ^ lA'sasP. (11) 

We see that a coupling IA333I ~ 0.5 would allow a branching ratio of about 20%. 

In Fig. ^ we show the magnitude for the branching fractions of the two-body decay for 
rrif = 70 and 100 GeV with m^o = 60 GeV and three-body decay for m^o = 60 and 80 GeV 
with rrif = 70 GeV, as a function of the coupling constant IA333I. We have chosen = 
mi^ = 200 GeV and (C^ + C^) ^ {C'^ + C'^) ^ 0.1, with A < 0.1, which enhances the three- 
body decay and is compatible with the masses we have assumed for the lightest neutralino. 
As one can see, the two-body decay can be very important reaching 10 — 20% or higher for 
large IA333I and therefore could play an important role in the top decay phenomenology. On 
the other hand, the three-body decay is too small to be significant. For illustration, we have 
also included the SM channel {t W~^b) in Fig. |^ at the top with = 70 GeV. 

III. SIGNATURES 

We base our analyses on the SM top-quark production at the Tevatron 

pp tlx, (12) 
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FIG. 2. i?-parity violating branching fractions for top decays as a function of the couphng 
constant IA333I. The two upper parallel curves represent the two-body decay t 6f+, for rrif = 70 
GeV (dashed) and nif = 100 GeV (solid) with rn^o = 60 GeV; while the two lower parallel curves 
stand for the three-body decay, t — > 6t+Xi) for "'-x? ~ GeV (dashed) and m^o = 80 GeV (solid) 
with ruf = 70 GeV. We assumed = m^^ = 200 GeV for the three-body decay. The dot-dashed 
curve at the top is for the SM channel {t — > W~^b) with mf = 70 GeV. 

from those recently reported in the CDF/DO experiments [jl3|. The ti signal can be identified 
above the SM backgrounds from their distinctive decay products. In our analyses, the event 
acceptance can be expressed by 

A= e,j-B,, 5,, (13) 

where Bsm and are the branching fractions of t and i decays to t ^ bW and t brxi, 
eij the corresponding detection efficiencies. 

With the interactions under consideration, the LSP Xi is not stable and will decay 
via Xi ~^ i^rbb. The collider phenomenology will then depend upon if it decays within the 
detector or not. The decay length is calculated by i = 7/5 cr, where the Xi life-time r can 
be written as [§] 
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FIG. 3. Estimated decay length for Xi ~^ Vr^h with various effective couphng k, assuming 
7/3 = 3. The dashed hne indicates the five-meter mark. 
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where the effective coupling k depends on the composition of the LSP and the masses of 
the virtual sfermions P,|l4|, and is typically n ~ 0.1 (100 GeV/m^)^ -^ass- the Tevatron 
energies, the 7/5 factor for x\ is typically about 1 ~ 5. With this estimation, we can calculate 
the Xi decay length as shown in Fig. |^. We see that for a rather light LSP m^o ~ (9(10 
GeV) or a weak effective coupling k < 10~^, x? Kiay escape from detection before it decays, 
resulting in missing energy as the standard SUSY signal. In fact, there is no mass constraint 
of this order for the LSP for general SUSY models. Also, such a small coupling is possible 
even with A333 ~ (9(0.1 — 1) if the third generation sfermions are very massive about 10 



TeV as advocated in certain "inverse hierarchy" models [|T^]. On the other hand, for a large 
region of the parameter space in (k, ttl^o), Xi will decay within the detector of a few meters. 
We will therefore study both cases below. 
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A. Long-lived Xi missing transverse energy 
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FIG. 4. Predicted number of events versus the coupling IA333I for Run I {^/s = 1.8 TeV) with 
the DO event reconstruction method and a data sample of 109.2 pb~^ for two representative values 
of ruf. The solid curves stand for a long-lived Xi ^^'^ the dashed ones for Xi to decay within 
detector. 

We consider the simple situation first: The LSP Xi is stable in the experimental scale, 
resulting in missing transverse energy. It is important to note that in this case the top-quark 
decay via the coupling A333 yields the same signal as for the charged Higgs boson search 
— > T^vh. The efficiencies for these final states are given by 



eij (%) t^hW t^hrv 
t bW 3.42 1.36 
t bru 1.36 0.41 



(15) 



We adopt the DO analyses and assume the same efficiencies for the event reconstruction. 
The predicted event rate can thus be obtained by 



= ^ a{tt) C + Nb, 
8 



(16) 



where a(ti) ^ 5.5 pb 0, £ is the integrated luminosity. A''^ presents the expected non- 
top-quark SM backgrounds, which includes VT-l-jets events and QCD multi-jet events with 
a misidentified lepton and large 0- Based on this event reconstruction scheme, we 
calculate the predicted ti signal events as shown by the solid curves in Fig. ^. We have 
included all the four channels listed in Eq. The SM expectation for tt events is implied 
at IA333I = 0. The larger the coupling is, the smaller the SM branching fraction (Bsm) 
would be, as seen from the top curve in Fig. Since the detection efficiencies are optimized 
for the SM decay channels, the total predicted number of ti events decreases for higher values 
of IA333I after convoluting with the efficiencies of Eq. ([I5|), resulting in a "disappearance" 
experiment. 

The non-observation of the signal beyond the SM at the Tevatron experiments can be 
translated to constraints on and IA333I. In Fig. |](a), we show the exclusion limit in the 
rrif — IA333I plane at the 95% confidence level. Here we have adopted the Poisson statistics 
treatment because the number of signal events with the existing data is not expected to be 
very large. Since our procedure closely follows the DO analysis we assume the DO data 
sample of 109.2 pb~^ and obtain the solid curve. The dashed curve is for 200 pb^^ (CDF 
and DO). The regions below the curves are excluded at a 95% CL. level. For the current 
mass limit TUf ^ 70 GeV |6|, we see that the existing Tevatron data can put a bound on 
the coupling to be IA333I < 0.94. The bound becomes weaker for a heavier m^. At Run II 
with ^/s = 2 TeV and a{tt) ^ 6.8 pb, the sensitivity is expected to be greatly enhanced. 
This is shown in Fig. |(a). For rrif = 70 GeV, the 2a bound is |A^33| < 0.38 (0.24) with 2 
(10) fb~^. This bound is essentially independent of the other SUSY parameters, and is more 
stringent than the indirect one obtained in Ref. P,p!0|. For comparison, the 3a contours are 
also given by the dashed curves in the figure. 
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FIG. 5. Contour plots for Run I in the mf — IA333I plane for exclusion at the 95% confidence 
level for the cases where (a) the Xi is long-lived and (b) the Xi decays within the detector. The 
solid curves are for the data sample of 109.2 pb~^ (DO); s^nd the dashed curves are for 200 pb~^. 
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FIG. 6. Contour plots in the mf — IA333I plane for exclusion at the 2a (solid) and 3a (dashed) 
limits at Run II with 2 and 10 fb~^, for the cases where (a) the Xi is long-lived and (b) the Xi 
decays within the detector. 
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B. xi ^ ^rbb 

As seen in Fig. ^, if m^o and k, are large Xi can decay within detector, yielding extra bb 
plus missing transverse energy in the final state. Two more b quarks would make the signal 
very different from the SM mode. This decay was considered at the Tevatron in the light of 



the total ti production cross section and a rather loose bound was inferred [0,^. We here 
consider two searching scenarios. 

First, we still adopt the disappearance method because the study of SM ti events is well 
established. Since the signal mode 

t T+bxl T+b bbur (17) 

is significantly different from the mode t —* rbv^ being searched for, the signal events can 
be effectively removed from the sample, leading to a lower acceptance efficiency. We thus 
keep the SM t bW mode with the same efficiency as in Eq. (p!5D, while take the others to 
be zero. This is an optimistic choice for the signal searches in a disappearance experiment, 
but it will not significantly alter the results. The "more disappearance" results in a larger 
deviation from the SM expectation of ti production, as shown by the dashed curves in Fig. ^. 
This in turn leads to more stringent constraints with the existing Tevatron data as shown 
in Fig. H(b) and with Run II data in Fig. |^(b). The current Tevatron data can put a bound 
on the coupling to be about IA333I < 0.63 for the short-lived x? with irtf ~ 70 GeV. At Run 
II, the 2a bound will be |A;533| < 0.29 (0.19) with 2 (10) fb^^. 

Alternatively, we could consider the direct search for the signal channel Eq. ([T7|) . To 
have a clean signal, one leptonic decay of t — blv {i = e, /i) is needed in combination with 



the signal mode. This ir channel has been recently studied experimentally in Ref. [jT7 
Unfortunately, the low branching fraction for the leptonic modes and the low detection 
efficiency for the r lepton render the signal events too low to be appreciable. Only with a 
luminosity significantly higher than the Run II projection could this search become possibly 
useful. We note that direct production processes pp tfX have been suggested to search 
for this interaction [l^ and it is worthwhile exploring further. 
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IV. DISCUSSIONS AND CONCLUSION 



Our analyses closely follow the study reported in Ref. for H"^ t^t-'t] while our signal 
for a long-lived x\ is for t^Xi- Although one may expect to have a softer r lepton in 

our signal than that in Ref. due to a more massive Xi; we would like to argue that the 
event reconstruction efficiency may be quite comparable since the signal under consideration 
carries larger missing energy, which should help in event selection. 

A more interesting question to ask would be that if there is a r^+missing energy signal 
established beyond the SM expectation, could we tell if it is from a decay or from f^? 
To properly address this question, one should realize that the I/ip interactions in Eq. @) 
violate lepton number. Therefore, and cannot be distinguishable in terms of the SM 
quantum numbers. One would need to specify the complete theory and work out the allowed 



state mixing |jT9[. In this sense, the Higgs fields lose the identity as that in /^-conserving 
SUSY theories. 

In conclusion, we have studied the top-quark decays t fb and t — > rbxi -R-parity 
violating interactions in SUSY theories. We found that the existing Tevatron data can put 
a 95% confidence level upper bound on the coupling to be 0.94 (0.63) for a long-lived (short 
-lived) Xi with = 70 GeV. At Tevatron Run II with an integrated luminosity of 2 (10) 
fb~^, one can obtain a 2a constraint as 0.38 (0.24) for a long-lived Xi ^i-nd 0.29 (0.19) for 
a short-lived Xi, beyond the current indirect limit. These limits from the direct search are 
largely independent of other SUSY parameters. 



Note added: After the submission of this paper, we received an article [20| in which hadronic 



observables of Z decays were studied and new constraint on IA333I was obtained. 
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